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Theory of Computation 1 


(( Hassan Kassim Mohammad 
Instructor: Hassan Kassim Mohammad 


Theory of computation 1s the theoretical study of capabılities and I1mitations of Computers (Theoretical 
models of computation). 


Objectives: 
Providing students with: 


o an understanding of basic concepts 1n the theory of computation through simple models of 
computational devices. 

o apply models 1n practice to solving problems 1n diverse areas such as string searching, pattern 
matching, cryptography, and language design; 

o understand the limitations of computing, the relative power of formal languages and the 1nherent 
complexity of many computational problems. 

o be familiar with standard tools and notation for formal reasoning about machines and programs. 


REFERENCES: 
1. Introduction to Computer Theory 2nd Edition 

Daniel I. A. Cohen John Wiley & Sons, Inc 1997. ISBN 0-471-13772-3 
2. Introduction to Automata Theory, Languages, and Computation, 2/E, 

John E. Hopcroft, Rajeev Motwanı, Jeffrey D.Ullman, Addison-Wesley 2001. ISBN 0-201-44124-1. 


Units: 6 


Grading Policy 


Attendance __ Assignments & Quizzes 


3 
misemeer 10 2g eg 
Final (0 |---| _ [0 


Notes 
Student must attend at least 80% of total classes to pass the course. 
Any kınd of cheating/plagiarism may result in a Fail grade 1n the course. 
No labs. But you should write some programs wıth any language you may know. 
There wıll be about 30 lectures 100 minutes each. 
Late homework submıss1ons wıll be penalized 


Office Hours 
Sunday, Monday, Tuesday, Wednesday 


Contact Information 


Office: computer science dept. room no. 67 
E-maıl: hassan.kassim @ yahoo.com 
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Syllabus 


sS Introduction, terminology, definitions 
2 Sets and operations I 
3 | lamguages 2إ‎ | 
| 4 1 Regular Expressions RE 4 
5 | 
5 
8 | 
5 | 


|| ¥ Finite Automata FA 

|| 6 ۹۹ Deterministic Finite Automaton DFA 

| 1 | Non Deterministic Finite Automaton NDFA 

| 8 Language Accepted by Finite Automata 

_ 9 | Convert Regular Expression into NFA || 
__ 10 || | Constructing regular expression from Finite Automata |__| 
T1 | Finite Automata with Epsilon moves || 
|_2 || | Moore and Mealy machines |9 | 
__ 13 __ | Converting between Moore and Mealy machine |__| 
_ 4 ___ | Pumping lemma for regular languages || 
(| KleenesTheorem |7| 
_ 16 | (Regular Grammar |0 | 
|_1 | Myhill-Nerode Theorem Minimization of DFA | | 
_|_____ _ (EXAM || 
_ 18 _ | Context-free Languages |3 | 
_ 19 | Pushdown Automata |7| 
_ 20 | (| CFGICFLtOPDA |18 | 
21 (PDAtOCFG/ICFL || 
_ 22 | CFG derivation trees Parsing 22| 
_ 23 | | Chomsky normal form |6 | 
__ 24 | | Greibach normal form |16 | 
_ 25 _ _ | Ambiguous CFLs || 
_|____ _ (EXAMÛ || 
_ 26 (TURINGMACHINES TÎ %4 | 
__ 21 | | COMPUTABILITY and COMPLEXITY | | 
__ 286 _ | Unsolvable Problems || 
_ 29 (TimeComplexity || 
_ 3 | CY¥Kalgorithm for CFO'S || 
__ 31 || | CFL pumping lemma and properties | | 
_ 32 | [Church Turing Thesis || 
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As a computer IT’, you must study the following: 
1- Automata and formal language. 
Which answers - What are computers (Or what are models of computers) 

2- Compatibility. 

Which answers - What can be computed by computers? 
3- Complexity. 

Which answers - What can be efficiently computed? 
In automata we wıll simulates parts of computers. Or we wıll make mathematical models of computers 
Automata are more powerful than any real computer because we can design any machine on papers that can 
do everything we want. 


Theory of computation is the theoretical study of capabilities and limitations of Computers 
(Theoretical models of computation). 


Sets 

Let A, B, and C be subsets of the universal set U 
Distributive properties 

An (BUCO =(An BJ U(ANn C 


AU(BNn O©O=(AUB)JnN (AUC 


Idempotent properties 
AN A=A, 


AUA=A. 


Double Complement property 
A ( = 


De Morgan ’s laws 
(AUB) =A n B 


(An BJ =A UB 


Commutative properties 
AN B=BN 4A, 


AUB=BUA. 


Associalive laws 
AN (Bn O=(ANn B)n C 
AU(BUOCO=(AUBUC 


Identity properties 
AU =A 


AN U=A4A. 
Complement properties 
AUA =U 

AN A” = O. 
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Language 


language 1s the set of all strings of terminal symbols derivable from alphabet. 


alphabet 1s a finite set of symbols. For example {0, I7} 1s an alphabet with two symbols, {a, b} 1s another 
alphabet with two symbols and English alphabet 1s also an alphabet. A string (also called a word) 1s a finite 
sequence of symbols of an alphabet. D, a and aabab are examples of string over alphabet {a, b} and O, 10 
and O01] are examples of string over alphabet {0, 7}, A null string 1s a string with no symbols, usually 
denoted by epsilon or lambda (A). A language 1s a set of strings over an alphabet. Thus {a, ab, baa} 1s a 
language (over alphabert {a,b }) and {0, II1} 1s a language (over alphabet {0,1}). The number of symbols 1n 
a string 1s called the length of the string. For a string w 1ts length 1s represented by lwl| . It can be The empty 
string (also called null string) 1t has no symbols. The empty string 1s denoted by A Thus IAl = 0. 


For example 00100| = 5, laabl = 3,1۸1 =0 


Language = alphabet + string (word) + grammar (rules, syntax) + operations on languages (concatenation, 
union, intersection, Kleene star) 


Kinds of languages: 
1- Talking language: (e.g.: English, Arabic): It has alphabet: > ={a,b,c,....z}From these alphabetic we 


make sentences that belong to the language. 
Now we want to know 1s th1s sentence 1s true or false so -We need a grammar. 
Alı 1s a clever student. (It 1s a sentence € English language.) 
2- Programming language: (e.g.: c++, Pascal):It has alphabetic:>={a,b,c,.Zz , A,B,C,..Z, 2,1, - ۸\.} 
From these alphabetic we make sentences that belong to programming language. 
Now we want to know 1f this sentence 1s true or false so we need a compiler to make sure that syntax 1s true. 
3- Formal language: (any language we want.) It has strings from these strings we make sentences that 
belong to this formal language. 
Now we want to know 1s this sentence 1s true or false so we need rules. 
Example: 

Alphabetic: >= {0, 1}. 

Sentences: 0000001, 1010101. 

Rules: Accept any sentence start with zero and refuse sentences that start with one. 

So we accept: 0000001 as a sentence satisfies the rules. 

And refuse: 1010101 as a sentence doesn't satisfy the rules. 


Example: 
Alphabetic: >= {a, b}. 
Sentences: ababaabb, bababbabb 
Rules: Accept any sentence start with a and refuse sentences that start with D. 
So we accept: aaaaabba as a sentence satisfies the rules. 
And refuse: baabbaab as a sentence doesn't satisfy the rules. 
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Regular Expression 
1s a set of symbols, Thus 1f alphabet= {a, b}, then aab, a, baba, bbbbb, and baaaaa would all be strings of 
symbols of alphabet. 
In addition we include an empty string denoted by A which has no symbols In it. 
Examples of Kleene star: 
1* 1s the set of strings {A, 1, 11, 111, 1111, 11111, etc. } 
(1100)* 1s the set of strings {A, 1100, 11001100, 110011001100, etc. } 
(00+11)* 1s the set of strings {epsilon, 00, 11, 0000, 0011, 1100, 1111, 000000, 000011, 001100, etc. } 
(0+1)* 1s all poss1ble strings of zeros and ones, often written as sigma * where sigma = {0, 1} 
(0+1)* (00+11) 1s all strings of zeros and ones that end wıth either 00 or 11. 
(w)+ 1s a shorthand for (w)(w)* w 1s any string or expression and the superscript plus, + 


1- Concatenation: 
Notation to the concatenation: . (The dot.): 
if LI = {x, xxx} and L2 = {xx} So (L1.L2) means L1 concatenated L2 and 1t 1s equal = {xxx, xxxxx } 


Examples on concatenations: 


Exl: 
LIND. 
LIZZ U, 


L1.L2 = {ac, ad, bC, bd} 
Note: ab differ from ba. 


EXZ2: 

2= {x}. 

L1 = {set of all odd words over >, wıth odd length}. 
L1 = {set of all even words over >, with odd length}. 
LIS {XXX XXX. KIKI oo }. 

LIZ UK RAR SAR 

L1.L2 = {x, XXX, XXXXX, XXXXXXX...}. 

Note: 

التكرار غير مسموح داخل المجموعة. 


Ex3: 
LI=S{S, KXXT. 
2= XE 


LILZSIRKS. ISX. 
Some rules on concatenation: 
A.X = XK 

L1.L2 = {set of elements } 


Definition of a Regular Expression 


A regular expression may be the null string, = 
A regular expression may be an element of the Input alphabet, r=a 
A regular expression may be the union of two regular express1ons, r=rl +2 


A regular expression may be the concatenation of two regular expressions, r=rl r2 
A regular expression may be the Kleene closure (star) of a regular expression r =r1* 
A regular expression may be a regular expression 1n parenthesis r = (r1) 
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epsı1lon 1s the zero length string 
0, 1, a, b, c, are symbols 1n sigma 
x 1s a varlable or regular expression 
( ... )(... ) 1S concatenation 
(... ) + ( ... J) 1S unlon 
( ... )* 1s the Kleene Closure = Kleene Star 


(A)(X) = (X)(A) =۸ 

(A)(X) = (X)( A) = x 

(A) + (X) = (x) + (A) = x 

e DED 

(A)* = (A)( A) =1 

)x)* + (۸) = )x(* = x* 

(x +۸)* = x* 

x* (a+b) + (a+b) = x* (a+b) 
XVE YY 

(x +۸)x* =x* (KK +۸) = x* 
(x+ ۸)(x+ ۸)* (x+ ۸) = x* 


A. is the null string (there are no symbols in this string) 
* 1s the set of all strings of length greater than or equal to 0 


Example: 
A = {a,b} // the alphabet 1s composed of a and b 


A* = {A, a,b,aa,ab,ba,bb,aaa,aab,... } 
The symbol * 1s called the Kleene star. 
O (empty set) 

A (empty string) 

( ) delimiter , 

u + unlon (selection) 

concatenation 


Given regular express1ons Xx and y, x + y 1s a regular express1on 


representing the set of all strings 1n either x or y (set unlon) 
x={a, b}, y= {c,d}, x+y = {a, b,c, d} 


Example 1 

Let A={0,1}, W1 = 00110011, W2 = 00000 
WIW2 = 0011001100000 

W2W1I = 0000000110011 

W1 ۸= W1 = 01 

A W2 = W2 = 00000 


x= {a, b}, y = {c, dQ}, xy = {ac, ad, bc, bd} 


Note: 
(a+b) =(aُ bb) 


Mustansıriya Unıiversıty — college of sciences — computer science department — second class 


Theory of Computation 7 (( Hassan Kassim Mohammad 


Examples of regular expressions 
Describe the language = what is the output (words, strings) of the following RE 


Regular expression output(set of strings) 


{A} 
{۸)} 
{aj} 
{ aa} 


{A, a, aa, aaa, ....} 


ا 
x‏ 


د 
ا 
x‏ 


{ a, aa, aaa, ... } 


{ a, aa, aaa, ...} 


ا 


{ ba, baa, baaa, ...} 
{ ba, baba, bababa, ...} 
a,b } 
a, A, b, bb, bbb, ... } 
A, a, b, aa, ab, ba, bb, ... } 


+ 
ba)+ 
alb) 


ا 


alb)* 


تآ | 7 7 —_jo‏ | ا حا کا کا کل سح 
o‏ 
£ 


aa(ba)*bb { aabb, aababb, aabababb, ... } 

+۵0 
a+ 
2 
+b+6 
a + b()* 1A, a, b, aa, bb, ab, ba, aaa, bbb, aab, bba, ....} 

5 
A + a) bc {bc, abc} 

ab* {a, ab, abb, abbb, ...} 

(A, ab, abab, ababab, ...) 

(x +2)( x +)* ( +2) 

ا 
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start with a 

end wıth b 

start with a and end wıth Db 
start with a or Db 


not start with b 


(b)* a (b)* a (b)* 
(a + b)* a (a + b)* a (a + b)* 
[(b)* a (b)* a (b)*] + [(a)* b (a)* b (a)*)] 


(b)* a (b)* a (b)* ]* 


Even number of 0’°s and contains 101 


Even number of 0’s or contains 101 


Exercise 


Ex. 1: Find a regular expression over the alphabet { a, b } that contain exactly three a's. 
Ex. 2: Find a regular expression over the alphabet { a, b } that end with ab. 
Ex. 3: Find a regular expression over the alphabet { a, b } that has length of 3. 


Ex. 4: Find a regular express1on over the alphabet { a, b } that contain exactly two successive a's. 
Ex. 5: Find the output (words) for the following regular expressions. 


0 
(x)* + (A) 
aa* b 


bba*a 


(a + b)* ba ا‎ 


(11 + 0(* )0+11(* 


01*+ 00+410 


(a+b)* abb+ 


)01+10)*11*00* | 


0+0*00 04 
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Finite Automata 
شكل رسومي يمتل تنقلات بين مجموعة حالات تسيطر عليها قواعد عن طريق شريط من المدخلات‎ 


1s a device consisting of a tape and a control circuit 
which satisfy the following conditions: 


1. The tape start from left end and extends to the right 


without an end. Head [Rzad only; 


2. The tape 1s divide into squares 1n each a symbol. 
3. The tape has a read only head. 


4. The head moves to the right one square every time 1t 
reads a symbol. It never moves to the left. When 1t 
sees no symbol, 1t stops and the automata terminates 
its operation. 


Finite ontrol 


5. There 1s a control determines the state of the 
automaton and also controls the movement of the head. 


Input Vat 


No 


recognize (accept) string ab 


A DFA represents a finite state machine that recognizes a RE. 


For example, the following FA: 


A finite automaton consists of a finite set of states, a set of transitions (moves), one start state, and a set of 
{ınal states (accepting states). In addition, a DFA has a unique transition for every state combination. 

1t 1S a set of states, and Its “control” moves from state to state 1n response to external “Inputs” . 

A fınıte automaton, FA, provides the simplest model of a computing device. It has a central processor of 
{ıinite capacity and 1t 1s based on the concept of state. 


finite state machine 1s a 5 tuple M = (O, A, T, S,F), where 
o Q --set of states = {qO, ql, q2, ....} 
o A -- set of input symbols ={a,b, ..., 0, 1, ...} 
o T --set of transitions or rules 
o 8S -- an Initial state 


o F -- the final state -- could be more than one final state 
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Designing (drawing) FA 
State Start Final Transition 
with numbers or any name | - or small arrow + or double circle (only one Input or symbol on the edge) 
a,b allowed means (a or b) 


State (q) Input (a) Input (Db) 


0 1 ر2‎ 
1 2 2 
ر‎ 2 9 
Transition diagram: 
TG has many inputs on the edge ab FA has only one 1nput on the edge a 


Deterministic Finite Automata DFA and Non Deterministic Finite Automata NFA 


DFA: different input from state to different states NFA: one input from state to different states 
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Language accepted by FA 


String 1s accepted by a FA 1f and only 1f the FA starting at the 1nıtial state and ends 1n an accepting state after 
reading the string. 


Examples of languages accepted by FA 
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| : 


(a+b)* b(a+b)* 


ab(a+b)* 


contains 3 a's b*ab*ab*ab* 


e of Computation 12‏ ججج 
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contains even number of a = (b*ab*ab*)+ 
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Convertinge Recular Expression Into a Finite Automata 
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(a+b)* b(a+b)* 


ab(a+b)* 


contains 3 as b*ab*ab*ab* 
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contains even number of a = (b*ab*ab*)+ 


divıdable by 3 


all bıt strings that begin wıth O and end with 1 


all bit strings whose number of O's 1s a multiple 
of 5 


all bıt strings with more 1's than O's 


all bıt strings with no consecutive 1's 
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Converting NFA into DEA 
Three steps : 1- find transition table ٽںږإقنتll عمل جدول‎ 
2- drawing new design رسم الشکل انفجديد‎ 
3- remove unreachable states lll Jوصولا إزالة الحالات الزائدة التي لا يمكن‎ 


Example : convert the following NDFA into DFA 


q1q2 | q2q3 | q233 


Note: Any state contains final mark 1t will be final state 
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Moore Machines 
Moore machine M 1s the Stuple M = (OQ, A, O, T, F, s) where 


O 1s a finite set of states 

A 1s the finite Input alphabet 

O 1s the finite output alphabet 

T 1s the transition function 

F is the output function Q2 A 

in addition to the start state or the Initial state 


A Moore machine 1s very similar to a Finite Automaton (FA), wıth a few key differences: 


e [It has no final states. 


e [It does not accept or reject Input, instead, 1t generates output from 1nput. 


e Moore machines cannot have nondetermıinistic states. 
Every Input gives output not 1f word belongs to the machine or language like FA 
In each state we stop we print out what 1ns1de that state (1t's content) 


so the output will be more than input by one because we start with start state and print out it's content before 
we trace the 1nput string 


This machine might be considered 
as a "counting" machine 


Input string  aaababbaabb 
State qO0q1q2q2q3q1q0q0q1q2q4340 
Output 000010000010 


this machine gives 1 after each aab 


aabaabaaababaab 
0001001000100001 


so we use Moore machine as a string recognizer to give us a mark (1) after each substring so we design a 
machine put O 1n all states except the one after the one represent end of substring aba 


The output of the machine contains 1 for each occurrence of the substring aab found 1n the input string. 
H.W. Construct a Moore machine that outputs a binary string that contains a 1 for every double letter 


substring 1n an input string composed of as and b’s. For example 1f abba 1s the Input string 0010 1s the 
corresponding output. 
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Mealy machines 
Moore machine M 1s a 5 tuple M = (OQ, A, O, T, F ,s) where 


O 1s a finite set of states 

A 1s the finite Input alphabet 

O 1s the finite output alphabet 

T 1s the transition function 

F is the output function Q2 A 

in addition to the start state or the initial state 


output on edge 
same input to output 


2/1 


aaabb 
01110 


Mealy machines are finite-state machines that act as transducers or translators, taking a string on an 1nput 
alphabet and producing a string of equal length on an output alphabet. we have r Dart 
Mealy machine does not accept or reject an input string, 7 


a Û 


we have 
just read bı 


b/1 


The machıne represented 1n below, outputs an E 1f the number of 1s read so far 1s even and an o 1f 1t 1s odd; 
for example, the translation of 11100101 1S OEOOOEEO. 


WE 2 0 


سے 
A Mealy machine that outputs 2 (‏ 
م 


E 1f the number of 1 1s even 
o 1f the number of 1 1s odd VF 


Binary inverter 


The following Mealy machine takes the one's 


complement of its binary input. In other words, 1t f1ps Output=11010 


Input = 0010 


each digit from a 0O to a 1 or from a 1 to a0. 


There are no accept states in a Mealy machine because 1t 1s not a language recogniser, 1t 1S an output 
producer. Its output will be the same length as 1ts input. 
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Binary Incrementer 


)0/0, 1/1 


One thing you wıll notice 1s the numbering of the 
states. Usually, 1f there are 3 states, we number them 00, 01, and 10. 
- the Input Dıt string 1s a binary number fed 1n backward 
- The output string will be the binary number that 1s one greater and that 1s generated right to left. 
- The machine wıll have 3 states: start, carry and no-carry. The carry state represents the overflow when two 
bits of 1’s are added, we print a O0 and we carry a 1. 
Let the input string be 1011 (binary representation of 11). 
e The string 1s fed Into the machine as 1101 (backwards). 
e The output will be 0011, which when reversed 1s 1100 and 1s the 
binary representation of 12. 
e In Mealy machine, output length = 1nput length. Hence, 1f input were 


1111, then output would be 0000 (overflow situation). 


Homework: 

Construct a Mealy machine that takes a string of as and b’s as Input and outputs a binary string wıth a 1 at 
the position of every second double letter. For example, for ababbaab the machine produces 00001010 
and for the input DDD the output string 011 is produced. 
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Kleene's Theorem 
Any language that can be defined by: Regular express1on/ Finite automata/ Transition graph 
Can be defined by all three methods. 


Proof 

There are three parts of our proof : 

Part1: every language that can be defined by a FA == can be defined by a TG. 
Part2: every language that can be defined by a TG == can be defined by a RE. 
Part3: every language that can be defined by a RE == can be defined by a FA. 


proof of part1 
Every FA 1s ıtself a TG. Therefore, any language that has been defined by a FA has already 


been defined by a TG. 


proof of part2 
The proof of this part wıll be by constructive algorithm. This means that we present a 


procedure that starts out with a TG and ends up with a RE that defines the same language. 


If we have many start states = become only one 


ا 
ا 


becomes 


If we have many final states = become only one 


اش 


becomes ` 


we are now going to build the RE that defines the same language as TG 
reduce the number of edges or states 1n each time 
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FI+rA+r3 


GE - © -~“-@‏ کک 
becomes‏ 2 ) 

© ira : 0 
becomes 


0 Ira" Ta : @ 


becomes 


becomes 


becomes 


special case : 


becomes r23 


Ara T3 


our goal: unique start state and unique final state. 
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Example 
Find the RE that defines the same language accepted by the following TG using Kleenes 


theorem. 


(aa + bb)(a + b)}*aa 


TT 
ت‎ 


a a 


{aa + bb}{a + b)*bb 


(aa+bbfarbj"aa + (aa+bbj(a+b}j™bb 


RE=(aa+bb)(a+b)*(aa+bb) 
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Example 


Find the RE that defines the same language accepted by the following TG using Kleenes 
theorem. 


(ab + ba) (aa + bb)*(ab + ba) 


(aarbkb}r(ab+tbalaatbb™tabrtbha) 


[taa+bb)+(ab+ba)(aa+bb)*(ab+ba)]* | 


RE= [(aa+bb)+(ab+ba)(aa+bb)*(ab+ba)]* 


Mustansıriya Unıiversıty — college of sciences — computer science department — second class 


Theory of Computation 25 Hassan Kassim Mohammad 


proof of part3 
Rule1: there 1s a FA that accepts any particular letter of the alphabet. 


There 1s an FA that accepts only the word A. 


FA accepts only A > FA accepts only a FA accepts 


a,. 


a+b > 


Rule2: 1f there 1s a FA called FA1, that accepts the language defined by the regular 
expression r1 and there 1s a FA called FA2, that accepts the language defined by the regular 
expression r2, then there 1s a FA calledFA3 that accepts language defined by the regular 
express1lon (rl +r2). 


Example 
We have FAI accepts all words wıth a double a 1n them, and FA2 accepts all words ending 


in b. we need to bu1lld FA3 that accepts all words that have double a or that end 1n Db. 


FA3 
Z1 =xloryl 
Z2 = x2 or y1 
Z3 = x1 or y2 
Z4 = x3 or y1 
Z5 = K3 Or y2 
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Example 


۲ : b 


-+y1 | 3 2 


ET 
Z2 Z4 Z5 


zl1=x1 or y1 
Z2=X2 Or y3 
Z3=x1 or y2 
Z4=x3 or y1 
Z5=x1 or y4 
ZzO6=x2 or y4 


21=%3 or y3 
28=3 Or y2 
292 or y2 
ry 
رای وا‎ 
ay 


0 ™ 

- 

١ E Se ۴ CC 

ك ا a‏ ا و 
E a o "AS : im,‏ 

E "ص‎ 0 LL °F ر‎ 

ا اا 1 EN‏ : 2 

ل TOs‏ إ اتل 1 ل 

LEN SO‏ ا ا 

ا ي ا ا 
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Home Work 


Let FA1 accepts all words ending 1n a, and let FA2 accepts all words with an odd number of 
letters (odd length). Build FA3 that accepts all words with odd length or end 1n a using 
Kleene's theorem. 


Home Work 


Let FAI accepts all words ending 1n a, and let FA2 accepts all words end wıth D. 


Buıld FA3 that accepts FA1+FA2 using Kleene's theorem. 
bı a 
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Rule3: 1f there 1s a FA1 that accepts the language defined by the regular expression r1 and a 
FA2 that accepts the language defined by the regular expression r2, then there 1s a FA3 that 
accepts the language defined by the concatenation r1r2. 

We can describe the algorithm for forming FA3 as follows: 

We make a z state for each none final x state in FA1. And for each final state in FA] we 
establish a z state that expresses the options that we are continuing on FA1 or are beginning 
on FAZ2. 


We have to connect (merge) the final state of FA1 wıth the start state of FA2 to produce new 
state (wich 1t 1s not final) 


Example: 


21< 


PA. 
221 gm 
Z4=y2 


But it 1s not simple like that, so we have to take all possablıites 
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Example 
We have FA[1 accepts all words with a double a in them, and FA2 accepts all words ending 
in b. we need to build FA3 that accepts all words that have double a and end with D. 
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Z1= x1 
AD. 

Z3= K3 or y1 
Z4= x3 Or y2 or 
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HomeWork 
Let FAI accepts all words with a double a in them, and let FA2 accepts all words with an 
odd number of letters (odd length). Build FA3 that accepts all words with odd length and 


have double a using Kleene's theorem. 
H a, 


Mustansıriya Unıiversıty — college of sciences — computer science department — second class 


Theory of Computation 31 (( Hassan Kassim Mohammad 


Rule4: if r 1s a regular expression and FA] accepts exactly the language defined by r, then 
there 1s an FA2 that wıll accept exactly the language defined by r*. 

We can describe the algorithm for forming FA2 as follows: 

Each z state corresponds to some collection of x states. We must remember each time we 
reach a final state 1t 1s possible that we have to start over again at x1. 

Remember that the start state must be the final state also. 


Example 
If r=a find r* 


Example 
If r=ab find r* 
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Example 
If we have FA1I that accepts the language defined by the regular expression: r=a*+aa™*b 


We want to build FA2 that accept the language defined by r*. 
Note: We will try to connect the final state with start state. 


Zz1=x1 

Z72=x4 

Z3=X2 Or x1 
Zz4=x1 Or x3 or x4 
Z5=x1 Or x2 or x4 
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Example 


Find FA2 that accept the language defined by r1* using Kleene's theorem. r1= aa*bb* 
2 : 


Z1=x1 
72 =2 
7 


Z4=x1 Or x4 


Z5=Xx2 Or X3 
Z6=x1] or x3 or x4 
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Problems 
For the followıIng transition graphs, find regular expression 


Consider following FA 


Find 
rl+r2 r2 + 13 rlr2 rlr3 fr2rl rlrl (r1)* (r2)* (rl+r2)* (rlr2)* 
(r2r1)* 
e Isrlr2=r2rl why ? 
e IiSrl +12=r2 +r] why ? 
e isrlrl = (r1)* why ? 
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Grammars 
A grammar 1s a set of rules which are used to construct a language (combine words to generate sentences). 


Sentence = noun verb noun 


Verb = went, eat, reading 
Noun = lesson, boy, school, book 
Boy reading book 


may be there are sentences have no meaning = book reading boy 


G=(N, T, S, P) 

N= set of nonterminal symbols (parts of speech (sentence, noun, Verb, ...))ex: S$ غير نهائية ويمكن اشتقاقها-حروف‎ 
كبيرة‎ 

T= set of terminal symbols (words, or symbols 1n) ex: نهائية ولا يمكن اشتقاقها-حروف صغيرة ج‎ 

S= start symbol non-terminal used to start every derivation. 


P= set of productions. سو‎ 


Example 


productions: 


نتبع البداية $ لنشتق الكلمات 
The derivation for aaaa 1S:‏ 


=< 5 
=< a8 
=< aaa8 
=< 5 
=< aaaa^ = 2 
RE= a+ 
Example 
productions: S 4 SS 
S د‎ a 
S4۸ 


5/2/1 جب SS‏ 3 يمكن كتابتها بهذا الشكل التبسبط 
Derivation of aa‏ 
S => SS‏ 

=< S99 

=> SSa 

=> SSSa 

=> SaSa 

=< a8a 

=< ۸2a = aa 
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Example 
S4aSlbSlalb 


Derive abbab 
SS => 
=> abS 
=> abbS 
=> abbas 
=> abbab 


Example 
S 2 aA/ bB 


A3>aS/a 
B32 bS/b 
Find bbaaaa 


Leftmost and rightmost derivation LMD RMD 


The leftmost nonterminal (LMN) 1n a working string 1s the first nontermıinal that we encounter when 
we scan the string from left to right. 

S ج‎ a8 | 

S5 

S =< aS 

S =< a25 

S =< aaaas 

S =< aaaaab 


Example 

Consider the Grammar G = (X, T, R, S) with X = {S, A, B, a, b}, T = {a, b} and productions 
S 3 AB 
A3 Aala 
B 3 bBa | ba 


Example 
S 3E 


EY E+TIT 
T9 1T*FIF 
F 3 (E) lid 


Converting Grammar into Regular Expression 
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Chomsky Normal Form CNFE 
Context free grammar (CFG) 1s the most 1mportant type of grammars because 1t 1s context 


free 1.e. the right hand s1de 1s contains anything from terminal and nontermınal so that 1t 1s 
wı1dely us1ng 1n the programming languages to represent the language rules and grammars. 
It wıll be difficult to deal wıth this type of grammars because the right hand s1de contains 
everything of terminals/nontermıinals, so Chomsky Introduced a new formula for constrain 
this grammar to be : 

The right hand s1de of a rule consısts of: t/ NN like the following grammar : 

S4 AB/BA/1 

A4 AA/a 

B4 BB/b 

Nonterminal 4 Nonterminal Nonterminal or Nonterminal 4 terminal 


The conversion takes place 1n four stages. 

Introduce a new start variable 1f the start 1n the right s1de. 

1. Eliminate lambda 

2. Elımınate all unıit-rules: rules of the form A 4 B 

3. Change the terminals into nonterminals 

4. Reduce rules with more than tow nonterminals into tow nonterminals 


Example: Convert the following grammar 1nto CNF: 
S 4 ASA laB 

A4 BIS 

B—4b|A 


S48 

S 4 ASAl| aB 
A—4 BIS 
B4b|^ 


Remove all epsilon productions, except from start variable. B ->e 
9S 49 

S 4ASAlaBla 

A4 BIS|1 

B—4b|l1 


Remove all epsilon productions, except from start variable. A -> 1 
9 5چ‎ 

S 4ASAlaBlalSAIASIS 

A4 B|IS|A 

B43>b 
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Remove unıt variable productions of the form S -> S 
S 4SIASAlaBlal SA| AS 

S 4ASAlaBlalSAIAS|IS 

A4 BIS 

B—4b 


Remove unit varlable productions of the form S5’ -> S 
S 4 SIASAlaBlal SA| AS 

S 4 ASAlaBlalSA|IAS 

A4 BIS 

B4 Db 


Remove unit variable productions of the form A -> B 
S 4 SIASAlaBlalSA| AS 

S 4 ASAlaBlalSAIAS 

A—4 BISIDb 

B4 Db 


Remove unit varlable productions of the form A -> S 
S 4 SIASAlaBlal SA| AS 

S 4 ASAlaBlalSA|IAS 

A4 SIbIASAlaBlalSAIAS 

B4 Db 


S4 SIASAlaBlalSAI AS 

S 4 ASAlaBlalSAI|I AS 
A—4 SIbIASAlaBlal SAI AS 
B4 b 


Add variables and dyadic variable rules to replace any longer productions. 
S4 AA1IUBlalSAIA®S 

S 4 AA1IUBlalSA|IAS 

A4 bIAAIIUBlalSAIA®S 

Al4 SA 

U +a 

B4 Db 
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Example: Convert the following grammar Into CNF: 

S 4bAlaB 

A¬4+4alaSl bAA 

B4blbSlaBB 

Step1: no lambda 

Step2: no unit production 

Step3: convert small 1nto capıtal 

S4 YA/XB 

A4a/ XS/ YAA 

B4b/ YS / XBB 

X4جa‎ 

¥ 4b 

Step4: convert more than 2 nonterminal into 2 nonterminal 

S4 YAIXB 

A—¬+al XS| YRI 

B4bl| YS| XR2 

X4 a 

¥ 4b 

RI 4 AA 

RZ 4+ BB 

Example: Convert the following grammar Into CNF: 
S 2 aSasS / SaSb /2 


S 2 SaS/SaSb/a/Sa/aS/ab/Sab/aSb 1 


S 2 SAS/SASB/a/SA/AS/AB/SAB/ASB 3 
A32a 
BZ2b 


S 2 SR1/SR2/a/SA/AS/AB/SR4/AR3 4 
R12 AS 

R22 AR3 

R32 SB 

R42 AB 
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Example 
E  4FE+TIE—-TIT 


T 3 T*FI T/FIF 
F 4 (E) lid 
Id 4alblc 


Then the string (a + b)*c belongs to above grammar and the derivation of this string: 


E4T 

4T *F 
4F *F 

4 (E) * F 
4 (E+T) *F 
4 (T+T) *F 
4 (F+T) *F 
4 (1d + T) * F 
4 (a+ T) * F 
4 (a+ FPF) * F 
4 (a + 1d) * F 
4 (a+ b) * F 
3 (a + b) * 1d 
3 (a+b) *c 


Derivation can also be nicely represented 1n a tree form, as bellow 


س ب چ س ٣‏ 


س إ1 سإ gı f‏ 


Derivation Tree for the Expression (a + bD)*c 


Mustansıriya Unıiversıty — college of sciences — computer science department — second class 


Hassan Kassim Mohammad 


The Internal nodes of the derivation, or syntax, tree are nonterminal symbols and the frontier of the tree 
consists of terminal symbols. The start symbol 1s the root and the derived symbols are nodes. The string 
(a + b)*c obtained from the concatenation of the leaf nodes together from left to right. 
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A correct parse of the string a + b*c as a sequence of shıift/reduce actions 1s given bellow. 


Action 
Shift 

Reduce 
Reduce 
Reduce 
Reduce 
Shift 

Shift 

Reduce 
Reduce 
Reduce 
Shift 

Reduce 
Reduce 
Reduce 
Reduce 


Accept 


Input 
a + b*c$ 
+ b*c$ 
+ b*c$ 
+ b*c$ 
+ b*c$ 
b*c$ 
*e$ 
*e$ 
*e$ 
*e$ 
c$ 


$ 
$ 
$ 
$ 
$ 


E$ 

+ EŞ 
b+ E$ 
id + E$ 
F + ES 
T+ EŞ 
*T + ES 
c*T + E$ 


id*T + E$ 
F*T + ES 


T+ ES 
E$ 


Parse of the expression a + b*c 
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Derivations and Parse Trees 

For every derivation there 1s a unique corresponding parse tree. 

A derivation 1s a leftmost derivation 1f the variable chosen for substitution, at any step, 1s the 
leftmost var1able. 


E4E+TIE-TIT we can write this grammar as:E E + EI|E-EIE*EIE/Elı1d 
T4 T*FIT/FIF 

F4 1d 

Derive 1d + 1d * 1d 

E4 E+T 

T+1T 

F+1T 

1d + T 

id + T * F 

ld +F * F 

id +1d * F 

1d + 1d * 1d 

The parse tree of an 1nput sequence according to a CFG 1s the tree of derivations. For 
example, the parse tree of 1d(x) + num(2) * 1d(y) 1s: 


CUCU 


So a parse tree has non-terminals for Internal nodes and terminals for leaves. 
There are tow types of parsing or derivation: 
Top down parsing and Bottom up parsing 
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Top-down parsing starts from the start symbol of the grammar S and applies derivations until 
the entire 1nput string 1s derived (1e, a sequence of terminals that matches the 1nput tokens). 
For example, 


E4E+T 

9 1+1 

4F F+T 

91+1 

SET FE 

41I+F*F 

41+1*F 

941+1 *1 
Which matches the Input sequence 1 + 1 * 1 


Bottom-up parsing starts from the input string and uses derivations 1n the opposıite directions 
(1e, by replacing the right hand side sequence of a production wıth the nontermınal. It stops 
when It derives the start symbol. For example, 
1+1 *1 
4F+1*1 
4T +1*1 
+4E+1*1 
E+ F*1 
4E+T *1 
9T *T 
SET 
3E 
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Ambiguity 


A CFG 1s amb1guous 1f 1t generates some string with more than one parse tree. 

A grammar 1s ambiguous 1f 1t has more than one parse tree for the same 1nput sequence. 
For example, the grammar G3 1s amb1guous since 1t has two parse trees. 

A string w 1s derived amb1guously 1n CFG G 1f 1t has two or more leftmost derivatlons. 
Suppose, that the rules of the expression grammar were written E 4E + E | E*E | 1d, then 
two different syntax trees are the result. If the first production E 4E + E were chosen then 
the result would be the tree on the left, On the other hand, choosing the production E 4E*E 
{ırst results 1n a ف‎ tree of an entirely different 


21% کے‎ 
l1 IS کار‎ | 


i E ۴ £ £ + E id 


E 


1 i i i 1 


t1‏ £ ظ 

Thus this grammar 1s ambiguous, because 1t 1s poss1ble to generate two different syntax trees 
for the express1on a + b*c. 
Exercises: Convert the following grammars into CNF: 
1. S 4aSal| bSbl al bl aa| bb 
2.9 4bAlaB 

A ¬bAAlaSla 

B 4aBB | bS|b 
3. S4Aba 

A 4aab 

B 4AC 
4. S 40A0 |1B1 | BB 

A ¬4€ 

B 48SIA 

C 1ا48‎ 
5. S 4aAa | bBbl| A 

A ¬4Cla 

B4C|Ib 

C 4CDE|^A 

D4A|IBlab 


Mustansıriya Universıty — college of sciences — computer science department — second class 


Hassan Kassim Mohammad 


Theory of Computation 45 


6. S—¬abAB, 
A—¬bABI|IA, 

B—4 BAalAlA, 

7. S¬+ABlaB 
A—¬aab|A 
B—+4bbA 

1- S—4+aSblab. 

2- S¬+a43A|IA 
A—¬abAlb 


3- S¬+abAB, 
A—¬bABIA, 
B— BAalAlA, 
5.9 4 aAD 
A 4 aB | DAB 
B 4b 
D4 d 
6. S 4 Aa|B 
B4 A| bb 
A—¬4al bc|B 
7.8 4 ASB |۸ 
A¬4aASla 
B 4 SbS | A| bb 


8. Show a derivation tree for the string aabbbb with the grammar 


S ك‎ ABIx 
A—ط‎ aB 
B—4 Sb 


9. wıth the following grammar Derive 1+(0+(1+0)-1) 


N4 N-N/N+N/(N)/D 


Dکج‎ 0/1 
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The Chomsky Hierarchy 


Noam Chomsky Introduced the Chomsky hierarchy which classifies grammars and languages. This hierarchy 
can be amended by different types of machines (or automata) which can recognize the appropriate class of 
languages. 


In the late fifties Noam Chomsky, a linguist at MIT’, was investigating the relationship between the syntactic 
structure of languages and the meaning (semantics) of statements 1n a language. 


The Chomsky hierarchy comprises four types of languages and their assoclated grammars and machines. 


Type Language Grammar Machine Example 
Type 3 Regular language Regular grammar RG E n a*b* 
Pushdown automaton 2 
Type 2 Context free language Context-free grammar CFG PDA a b 
Tvoe 1 Context sens1t1ve Context sensitive grammar Linear bounded automaton | #4 2# 4# 
3P language CSG LBA a b c 

Recurs1vel Turing machine 

Type 0 3 Unrestricted grammar UG ح‎ computable 
enumerable language TM n 


regular languages _context-free languages *_context-sensitive languages %_recursive enumerable 
languages. 
Type 3 type2 typel typeÛ 


All languages 
Type O or recursively enumerable languages 
Decidable languages 


Turing machines 


Type I or context sensitive 
languages 


Type 2 or context free 
languages 


pushdown automata 


Type 3 or 
regular languages 


finite automata 


Mustansıriya Universıty — college of sciences — computer science department — second class 


Theory of Computation 47 (( Hassan Kassim Mohammad 


Type 3: 

regular grammars generate the regular languages. Such al11 grammar restricts 1ts rules to a single 
nonterminal on the left-hand side and a right-hand side consisting of a single terminal, possibly followed by 
a single nonterminal. The rule S 4 A 1s also here allowed 1f S does not appear on the right side of any rule. 
These languages are exactly all languages that can be decided by a finite state automaton. Additionally, this 
family of formal languages can be obtained by regular expressions. Regular languages are commonly used to 
define search patterns and the lexical structure of programming languages. 

N 2t/N 

A 2> a/aB 

S > aS/b 

Problem is left recursion A > Aa 


Type 2: 

context-free grammars generate the context-free languages. These are defined by rules of the form 
A —4 + wıth A a nontermınal and y a string of terminals and nonterminals. These languages are exactly all 
languages that can be recognized by a non-determıinistic pushdown automaton. Context free languages are 
the theoretical bas1s for the syntax of most programming languages. 

S 2 (N U t)* 

S 2 SS/aA/bA 

S21 

S2abB 


Type 1: 

context-sensitive grammars generate the context-sensıtive languages. These grammars have rules of 
the form aA ¬4 ayPB with A a nonterminal and a, B and Yy strings of terminals and nonterminals. The strings O 
and may be empty, but y must be nonempty. The rule S ك4‎ gE 1s allowed 1f S does not appear on the right 
side of any rule. The languages described by these grammars are exactly all languages that can be recognized 
by a non-determınıstic Turing machine whose tape 1s bounded by a constant times the length of the Input. 

U 2V U,V (N U t+ 

S2SS 

aA Z2bAa 

BB 2aB 

A321 wrong 

Left side <= right side 


Type 0: 

unrestricted grammars Include all formal grammars. They generate exactly all languages that can be 
recognized by a Turing machine. The language that 1s recognized by a Turing machine 1s defined as all the 
strings on which 1t halts. These languages are also known as the recursively enumerable languages. Note that 
this 1s different from the recursive languages which can be decided by an always halting Turing machine. 

U 2V U,V (N U1)* 

S2SS 

S2aAb 

aA 2Aa 

BBZ2a 

aA ZbAa 

Ba 2bAb 
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Push Down Automata PDA Pushdown Automaton -- PDA 


PDA consists of 7 components (7-tuple): Input String 
M = (Q, Sigma, Gamma, delta, qÛ, ZO, F) where 


Q = a finite set of states 

Sıgma = a finite alphabet of input symbols (input tape) 
Gamma = a finite set of push down stack symbols States 
Delta = a group of transitions 
qO = the ıInıtial state 

ZO = the initial stack contents - stack symbol A 6 
F = the set of final, accepting, states 


| | | Stack 


| 


N 


It 1s the machine that accepts CFG languages, the most Important form 1s anbn , n>=1 1.e there 1s a relation 
between exponents of both variables and It 1s gives strings like aaabbbA, aaaaabbbbbA , aaaaxaaaaA, abbcbbaA 
Consist of basic shapes to represent PDA like triangle, diamond, trapezoidal ... 


The START symbol Just points to the start state 
The ACCEPT symbol represents a final state or accept 
The REJECT symbol represents a reject state or error 


Some of states are decision state; 1t 1s represent every function performed 1n a state by a different type of 
box. The typical task performed 1n a state 1s to "read and branch" which wıll now be represented by a 
diamond shaped box, such as: 


PUSH 
b a 
A 


Then we have two main operations e1lther push the input string into the stack or pop 1t from the stack depending 
on the reading string We need a type of memory which 1s the stack (First In Last Out) contain A simple to 
represent the empty stack 
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We have to push the first part of string Into the stack and then pop contents of the stack when we start 
reading the second part of the string 

For example aaabbbA We wıll push all "aaa" into the stack then we wıll pop when we start reading "bbb" 
We can div1de string reading Into tow stages 

When we get the first part "aaa" we wıll push them into the stack 

and when we read the second part "bbb" we wıll pop from the stack, we should get "aaa" and the stack wıll 
be empty and the string 1s reached to the space symbol 

so 1f we read the space symbol we have to pop from the stack, 1f we got space symbol that 1s means the 
string 1s accepted 


Iracing the input string on the PDA 
We wıll trace and witching 3 variables state, stack and the tape (1input string) 
For example: trace aaabbbA on the PDA a“ b" n>=1 


State Stack Tape ۰ 

START ۸ aaabbbA oS. 

READ ۸ aaabbbA ر‎ 

PUSH aA aaabbbA ۰ 

READ aA aaabbbA کک‎ 

PUSH aaA aaabbbA | 

READ aaA aaabbbA ا‎ 

PUSH aaaA aaa bbbA ) 

READ aaaA aaabbbA hb. A ۸ 

POP aaA aaabbbA ف‎ a, | 
READ aaA aaabbbA 

POP aA aaabbbA a 1 | 
ë1 
READ ۸ aaabbbA ) 
POP - aaabbbA 

ACCEPT 
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یمکن تصميم P0۸‏ لاي تعبیر RE‏ وذلك من خلال قور أءة المدخ<ات فقط بدون ادخالها الى المكدس اي قر اأءة فقط 
Example: a‏ 
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Example a + b 
Example (a + b)* 
Example a* b 


Example a b* 


H.W. Design a PDA for a*b* 


H.W. Design a PDA for a" b" n>=1 


H.W. Design a PDA for ca db a n>=1 


H.W.a bcd n>=1 
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We can design a PDA to check the odd palindrome (string can be read from right or left) RADAR, MADAM 


Palindrome 


We wıll use X as a mark to distinguish the middle of the string 
We wıll push all letters of the first part before X 


Read x with no action 
Then we will pop the contents of stack 


Check it with the tape, 1f 1t 1s same 
We continue pop and read, 


Until we get A then we will pop e 


One time 1f 1t 1s A so we reached سی پا‎ 


العلامة المميزة × التي تفصل الجزئين 


ACCEPT state else go to Reject state 


ACCEPT 
Example : aabaXxabaaA و‎ 


State Stack Tape 
START A aaaxbbbA 
READ A aaaxbbbA 
PUSH aA aaaxbbbA 
READ aA aaaxbbbA 
PUSH aaA aadaxbbbA 
READ aaA aaaxbbbA 
PUSH aaaA aaaxbDbbA 
READ aaaA aaaxbDbbA 
READ aaaA aaaxbbbA 
POP aaA aaaxbbbA 
READ aaA aaxabbbA 
POP aA aaaxbbbA 
READ aA aaaxbbbA 
POP ۸ aaaxbbbA 
READ ۸ aaaxbbbA 
POP - aaaxbbbA 
ACCEPT 
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Non deterministic palindrome NPDA 


Here there is no distinguish mark, 
but our palindrome 1s odd 
(there 1s (a or b) 1n the middle) 

| . ت‎ 
then continue reading the tape PUSHa ) 
and pop from the stack 


so we wıll push all letters (a and b) 
until we get A 


and then read one (a or b) 


د 
ACCEPT J]‏ سر POP‏ 


H.W. 

Trace the string abbababbaA on the above PDA 
State Stack Tape 

START A abbababbaA 
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H.W. a" b" n>=] 3 aaaaaabbbA 


` vs [ START 


2 a a . 
. READ, ٠ POP, ا ا‎ ۱ 
: ك‎ 
| PUSH a ACCEPT CD 


State Stack Tape 
START A aaaaaabbbA 
H.W. 


Design a PDA for a2nbnamb2m n,m>=1 
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Turing Machine, TM 


A Turing machine 1s defined by M = (Q, Sigma, Gamma, delta, qÛ, B, F) where 
Q = finite set of states including qO 

S1gma = finite set of Input symbols not including B 

Gamma = finite set of tape symbols 1ncluding S1gma and B 

delta = transitions mapping Q x Gamma to Q x Gamma x {L,R} 

q0 = initial state 

B  =blank tape symbol, initially on all tape not used for input 

F =setof final states 


M = ( Q, Sigma, Gamma, delta, qO, B, F) 


رهی اا كر ار ال ركا كع ارف ان و السار راك ن رما على فك ات م اس ال کت 
Sg a OS‏ 
ونقوم بقراءة الحرف ونقوم بتغييره الى حرف كبير او نبقيه كما هو ونتجه لليمين او لليسار 


It 1s designed to solve 3 or more of letters with same no of letters abc .... And 1t can be designed for any 
type of grammars 

Read write direction 
Input output direction 
Tape same/different R/L 


(Ab L) 
(aa, l) 


1 START 


ë HALT 


(ir.r, F) 
(a+b)b(a+b)* (a,a.R) (h.b. t) 


(hh) 


(| START] 


(b.b.K) 
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tape 
aaabbbA 
AaabbbA 
AaabbbA 
AaabbbA 
AaaBbbA 
AaaBbbA 
AaaBbbA 
AaaBbbA 


AAaBbbA 


هنا نريد ان نتاكد من الكلمة هل يمكن قراءتها من اليمين واليسار 
حيث سنقوم بتاشير اول حرف مع الاخير والتاني مع قبل الاخير 
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ab" ,n>=1  aaabbbA 


aa,f) 


پک پم 


Palindrome 


aabaabaaA 


وهکذا 


(b,b,R) 
(a,a, f) ° 


1 START 


H.W. 


Design a Turing machine for 


a bc" ,n>=1 


Mustansıriya Unıiversıty — college of sciences — computer science department — second class 


Hassan Kassim Mohammad 


Theory of Computation 56 


Mustansıriya Unıiversıty — college of sciences — computer science department — second class 


